Improve efficiency of new_dynamic by catching IdenticalZero
and IdenticalOne
cases
#232
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR should improve efficiency of
new_dynamic
. Many of the basic math ops such asadd
,add_eq
,sub
,sub_eq
,mul
,mul_eq
,div
,div_eq
, andazmul
catch cases where one argument is a CppADConstant
and has value either 0 or 1, when this allows avoiding placing an operation on the AD tape. For example y = x + c where c is identically 0 does not require an addition op to be placed on the tape.However, such checking was not in place for cases where neither argument is a CppAD
Variable
but rather at least one of them is a CppADDynamic
. This meant that the operation tape used for updating dynamics (vianew_dynamic
) could include potentially many trivial operations such as adding 0 or multiplying by 1. In a simple example such as double-taping of second derivatives of a quadratic form such as in a multivariate normal density calculation (e.g.x' A x
, whereA
is dynamic and derivatives are with respect tox
) could generate a mostly wastefulnew_dynamic
tape.This PR attempts to imitate the logic used for the
Variable
cases to handle similar cases forDynamic
. In my test problem, it fixes a memory explosion that was occurring. All the ops listed above were modified.This PR is likely not ready to merge because I am not sure how to add to CppAD's test suite. @bradbell is that something you could do, or provide guidance on? I am not clear if there is comprehensive testing of
Dynamic
operations or other similar place that might be obvious to extend. My own motivating problem only touches a couple of these ops, so the others remain untested. I am making this PR now to see if this looks on the right track to you before spending more time on it.I note that
pow
could also receive the same kind of extension as the other ops listed above. However, I noticed that for theVariable
cases, it checks forif( IdenticalZero( y.value_ ) )
andif( IdenticalZero( x.value_ ) )
, so it is not checking ify
orx
are constants the way the code for the other ops does. E.g. in the other ops, there would be something likeif( (!dyn_y) && IdenticalZero( y.value_ ) )
. Any chance this is a bug forpow
?Also @bradbell you told me to be sure I could run
bin/check_all.sh
. That has been pretty hard to get fully working. I believe all the code checks run, andmake check
(called frombuild
) passes. Butbin/check_all.sh
gets stuck in a documentation step and it's been a bit of an obstacle.